Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, an image pickup apparatus, and a storage medium which are capable of reducing the influence of hand shake of a user, shake of a subject and the like on an image or a video when an image or a video is captured.
Description of the Related Art
There are electronic shake correction of performing geometrical deformation of images, and optical shake correction in image pickup apparatuses. The latter includes in-lens shake correction which drives some lenses in an imaging optical system and imager shake correction which moves an imaging element. None of these shake correction methods is able to cancel out variation in parallax differences due to shifting of the viewpoint in image shake caused by movement of the position of an image pickup apparatus. Meanwhile, image pickup apparatuses having a light field photography function, a pupil division photography function and the like and capable of capturing a plurality of viewpoint images can perform a ranging operation. A trajectory of an image pickup apparatus including positional deflection in addition to rotational deflection of the image pickup apparatus can be correctly measured using a distance acquired through a ranging operation (N. Engelhard et al., “Real-time 3D visual SLAM with a hand-held RGB-D camera.” European Robotics Forum, 2011). Japanese Patent Laid-Open No. 2012-191351 discloses a technique of generating a virtual viewpoint image by controlling a mixture ratio between parallax images. Japanese Patent Laid-Open No. 2016-42662 discloses a technique of canceling out variation in parallax differences due to shifting of the viewpoint to perform shake correction.
However, imaging unit capable of performing shake correction and correction for shake of subjects are limited, and image shake called shake cannot be corrected using only the method of controlling a mixture ratio between parallax images and performing shake correction according to shifting of the viewpoint of an image. Furthermore, when a method of simply combining image compositing according to mixture ratio control and conventional shake correction is used, excess correction may occur and thus unnecessary image shake may be added.